Cost-Effectiveness Analysis and the Formulary Decision-Making Process

BACKGROUND: Faced with high drug expenditures in an environment of cost containment, drug formulary systems, particularly in managed care, have become more dependent on pharmacoeconomic evaluations to assess the value of new products. Within pharmacoeconomics (PE), cost-effectiveness analysis (CEA) is the most commonly used method. However, current methodological concerns about CEA have limited its practical contribution to the formulary process. Advances in analysis are likely to improve the relevance of CEA over time. OBJECTIVES: The purpose of this paper is to review CEA, its limitations, and its applications in formulary decision making in order to promote greater utility of CEA for managed care pharmacists. SUMMARY: Enhancements to CEA, such as the development of modeling software, rank-order stability analysis, cost-consequence analysis (CCA), and budget impact analysis are discussed. A combined method of CCA-CEA and standardized guidelines are suggested to improve the impact of CEA in the drug formulary process. CONCLUSIONS: Along with advances in its methodology and relevant standardized guidelines, CEA will gain increased importance in formulary decision making, helping to assure the goal of cost containment while ensuring quality of care.

impact of CEA or how CEA results are implemented in actual settings. For example, a recent article by Carey et al. asserted that pharmacoeconomics (PE) has already slowed down the rise in drug costs for insurers and other payers, but no supporting evidence was provided. 4 The purpose of this article is to review and evaluate CEA in terms of its application in formulary decision making and to examine available analytic tools that may facilitate formulary decisions.
ss The Formulary Decision-Making Process 8 The general procedure of adding a drug to the formulary is described in 7 typical steps by Glennie et al. 8 (Figure 1): 1. Pharmacological and clinical evaluation. A detailed clinical justification for the new drug is first submitted by the potential prescribers or the manufacturer to the P&T committee, which is composed of pharmacists and physicians and, in some cases, administrators. Subcommittees may be needed to evaluate hightech or specialty drugs. At this initial stage, only clinical decisions concerning the use of the drug are usually addressed. Drug information (such as new ingredients, efficacy, safety, tolerability, dosage and route of administration, ease of use, patient acceptance, etc.) from clinical trials as well as literature reviews is evaluated at this stage. 2. Pharmacoeconomic evaluation. After pharmacological and clinical evaluations, the costs and economic benefits of the drug are addressed. This is the step where PE tools may be applied. Optimally, the PE evaluation would draw on published data, noting authorship, funding sources, methodologies, etc., and be assessed by persons with an understanding of PE. The P&T committee may contract with knowledgeable pharmacoeconomists for consultation when necessary to evaluate the data produced by the pharmaceutical companies. Experience with economic modeling is crucial to examine whether assumptions put forth by a drug company in a given model are reasonable and applicable to the specific population served by the provider organization. The fit is never perfect, and pharmacy managers often need to spend time discerning the value of drug company-offered models. Here, various data sources can be employed (e.g., databases of pharmacy, medical, and laboratory claims; PE literature; expert panels; experienced real-world practitioners), where the time spent in acquiring data should be proportional to the magnitude of the decision at hand by the P&T committee. 3. Development of drug-use criteria. Criteria should be developed with the goal of ensuring appropriate drug use in the covered population over time. The clinical evaluation and PE evaluation mentioned above are useful in helping the P&T committees develop specific criteria. Managed care tools such as prior authorization, quantity limits, prescriber edits, or education interventions with prescribers may be applied, as well as drug benefit design, to promote appropri-ate use of the adopted new drug. 4. Approval by the P&T committee. Once the clinical and PE evaluations are completed and the criteria for the use of the drug have been developed, the compiled material should be reviewed and adopted or rejected by the P&T committee. P&T committee recommendations are typically disseminated to the medical and pharmacy staff. 5. Administrative and ethical reviews. Administrative reviews, though important, tend to be relatively straightforward in terms of what is being examined, such as the terms of rebate contracts with drug manufacturers. As a rule, the health plan is responsible for providing summary information to the manufacturers about the principal drug benefit plan parameters such as copay tiers and the number of health plan members subject to each benefit design. Ethical reviews may include consideration of the type and scope of claim-level data required by the rebate or discount contract with the pharmaceutical manufacturer. Hence, the drug discountrebate contract process requires careful review that includes coordination of the P&T committee process carried through to the administrative level. 6. Drug-use monitoring. Structured monitoring of drug use and a plan for follow-up evaluation is an inherent part of the P&T committee' s responsibility to optimize drug use. At the time of adoption, plans should be put in place for such continuous quality improvement reviews of drug claims data, including the need for ongoing drug utilization reviews. 7. Follow-up review by the P&T committee. Following www.amcp.org Vol. 10  Developed by Zhixiao Wang, based on the procedure proposed by Glennie JL et al. 8 approval, formulary decisions should be subject to a dynamic process of constant oversight. Effective formulary management requires the P&T committee to review the results of retrospective or prospective audits to ensure the appropriate and effective use of drugs.
ss Cost-Effectiveness Analysis and Its Application in the Formulary Process CEA has been the most commonly used PE method in drug evaluation. 9 It can help decision makers quantify the value of competing interventions and maximize efficacy of care. However, it also tends to rely on data and assumptions regarding costs and effects that can be manipulated to make a product look better. Although certain countries have formalized guidelines for CEA, there is not a universally accepted approach to this PE technique in the United States. The U.S. Food and Drug Administration (FDA) does not require CEA in drug approvals, but, increasingly, provider organizations seek such data along more standardized formats, as recommended in the AMCP Guidance for Submission of Clinical and Economic Evaluation Data to Support Formulary Listing in U.S. Health Plans and Pharmacy Benefits Management Organizations. 10 Issues about guidelines will be discussed later in this article.
In CEA, costs are measured in dollars and then compared with the effects or improvements of treatments, which are measured in various "natural" or constructed units, such as quality-of-life measures. CEA is most appropriate when the alternative therapies result in different levels of a common effect, such as 2 asthma medications that result in "symptom-free days." 11 The CEA is constructed to identify the most cost-effective therapy when the goal is to provide the highest-quality pharmaceutical care within a fixed budget. To be informative, the drug alternatives in the analysis should include all reasonable options and a baseline comparator (which should reflect the current practice and use a drug on the current formulary). 12 For example, when P&T committees consider new drugs, such as the COX-2 drugs celecoxib, rofecoxib, or valecoxib for formulary inclusion, the COX-1 inhibitors (NSAIDs) should be considered as alternative therapy.
The effect measured should be the primary outcome of the treatment. It is desirable to use final outcomes such as lives saved, life-years saved, cases prevented, rates of a specific side effect, etc. Sometimes, intermediate outcomes can be used if the relationship between the intermediate and final outcome measure can be estimated. For example, the intermediate outcomes of reduced cholesterol levels or reduced blood pressure can be used because they predict future health outcomes, including reduction in cardiac risk.
Measured costs and effects are calculated within realistic clinical pathways, along with probabilities of patients going down each of the clinical paths. The probabilities and paths are used, within some form of model of uncertainty such as a deci-sion tree or a Markov model, to calculate population-expected values. Markov models are different from decision trees in that Markov models can simulate the uncertainty from repeated events, such as complications in the chronic conditions, by putting patients in a finite number of mutually exclusive and exhaustive health states; the probability of transition between health states depends on the current status. Decision trees are only appropriate for simulating uncertainty of nonrecurrent chance events and outcomes. Then, usually, 2 forms of ratios are calculated. The first type of ratio is an average cost-effectiveness ratio, which is defined as the mean value of the costs divided by the mean value of effect for each alternative treatment. Average cost-effectiveness ratios (change in cost divided by change in effect) can be useful in considering the overall affordability of an intervention.
However, average ratios can be misleading when making a decision between 2 exclusive treatments paid for out of the same budget because they do not provide direct information about the costs and effects of making such a decision. For example, in a cost-effectiveness study about interferon beta in multiple sclerosis relative to usual care in the health care setting of the United Kingdom (UK), the average cost per QALY gained with interferon beta was £7,852.3 (pounds sterling) while the average cost per QALY with the usual care was £2,056.8. 13 All that one can tell from this information is that the average cost for interferon beta was higher than that of usual care. Upon closer inspection, discussed in the next paragraph, the average cost-effectiveness ratio is found to be a poor approximation of the actual trade-off relevant to the decision. The reason is that average ratios are a comparison with no treatment, while the relevant and more important decision is between the new treatment and the existing standard of care.
This second type of ratio is called an incremental cost-effectiveness ratio (ICER), which is the change in costs divided by the change in effects in moving from a lower-cost/lower-effect treatment to a higher-cost/higher-effect treatment. Incremental ratios provide an estimate of the cost corresponding to a change in the measured effect through a change in drug therapy and thereby provide information regarding the relative efficiency of alternative options. In the above study involving interferon beta, the ICER was £51,582 per QALY in comparing interferon beta with usual care, which indicated that an additional £51,582 has to be spent to gain 1 additional QALY when switching from usual care to interferon beta. 13 Such a high ICER may not be justified as a costeffective preventive treatment for multiple sclerosis from the National Health Service perspective in the UK.
Finally, sensitivity analysis should be conducted to test the robustness of the results. CEA studies are inherently based on assumptions, and these assumptions often reflect a certain degree of uncertainty. Moreover, values of some variables in CEA are very difficult to measure with great accuracy, or they change over time and in different settings. By changing the val-ues of these variables over a certain range or by changing the assumptions of the CEA model, sensitivity analysis provides insight into the robustness of results. Sensitivity analysis also assists in identifying variables that may have a large impact on the results from the CEA model. For example, in a study comparing nefazodone and imipramine (antidepressants), sensitivity analysis indicated that the cost-effectiveness model was most sensitive to assumptions on treatment compliance rates. 14 The ICER ranged from $2,572 to $5,096 per QALY gained when varying compliance rates while the base case ICER was $4,065 per QALY gained. This gives readers a better sense of how to generalize the ICER results and what to expect about the ICER of nefazodone relative to imipramine in situations where compliance rates are different from the base estimate. For example, if the entire range is considered a good value for increased health, then one can be more confident about adopting the new treatment into a patient population with unknown compliance.

ss Application of Cost-Effectiveness Analysis in the Formulary Process
If used properly, CEA can facilitate formulary decision making for drugs within the same class or with a common effect (e.g., cholesterol reduction) within a relatively homogenous population. It can also be used to evaluate the economic impact of a formulary decision if head-to-head (i.e., comparing a treatment with the next best available treatment) data are available. Some studies have addressed the application of CEA in the formulary decision-making process, though not all of them were positive. For example, a survey of 103 hospitals conducted in 1995 regarding the use of CEA in a hospital formulary decision revealed that CEA was only "a minor tool" in the decision making. 15 The most commonly stated barriers to effective use of CEA included lack of timeliness of studies, lack of generalization on hospitalized patients, biased industry sponsorship, and lack of expertise on economic evaluation.
We examined the potential impact of CEA by researching selected studies. The literature search was performed in July 2001 on MEDLINE and Ovid, 2 large electronic journal databases that provided access to a range of bibliographical or fulltext biomedical databases, using different combinations of keywords of "pharmacoeconomics," "cost-effective analysis," "formulary," and "formulary decision." The objective of this literature search was to identify typical studies that reflected the potential benefits of CEA as well as current levels of CEA utilization on formulary decisions. A total of 62 abstracts were identified, and 22 of them were found to be relevant to the application of PE research in formulary decisions. Original articles were obtained and analyzed. Most articles were academic studies rather than examples of how the results of CEAs actually assisted formulary decisions to achieve more cost-effective use of drug therapies.
Cohen studied the application of CEA in the treatment of depression and concluded that the total cost of disease management is similar for generic tricyclics and the more expensive selective serotonin reuptake inhibitors. 16 This result was due to the higher costs of related resource utilization such as outpatient visits and hospitalizations associated with tricyclics that offset their advantage in lower drug acquisition cost. Cohen suggested that CEA in formulary decision making would optimize the use of overall health care resources. This is the key since drug-use costs include more than just the direct acquisition cost of the drug itself.
McCoy et al. did a CEA to assess a 1995 formulary decision that designated cimetidine as the primary histamine-2 receptor antagonist (H2RA) and restricted the use of famotidine. 17 The study used a decision-tree model to estimate the average direct medical costs for the 2 treatments in a 2-month period. The results showed that the average cost of receiving cimetidine was $82.01 and the average cost of famotidine therapy was $92.45, while treatment success rates, the common efficacy measure, were identical for cimetidine and famotidine. The study supported the formulary decision at the health care institution. Therefore, retrospective CEA studies can serve as a measure of evaluation of past formulary decisions.
An important cost category of drugs is antibacterial drugs, which account, generally, for 6% to 21% of a drug budget, or 3% to 25% of the total prescription market in various countries. 18 Hillman discussed the role of CEA in the development and acceptance into formularies of new oral antibiotic products. 18 The author suggested that a comprehensive CEA should be developed by comparing the total stream of costs of an intervention with the total stream of outcomes, with a long-term follow-up. Such analyses would disclose the treatment alternative' s hidden costs and real benefits. The author also pointed out that the challenge for the development of new antibiotics is to balance patient needs, such as convenience for administration, safety, and a broad-spectrum of activity, with the economic needs of society within the cost-effectiveness perspective, that is, efficient use of limited resources to achieve maximum benefits.
Finally, in a look to the near future, biotechnology products used for the treatment of cancer patients have already reached the marketplace. Generally, they are very expensive because of the high input of capital and other resources consumed in research and development. For example, imatinib mesylate (Gleevec), a new drug for the treatment of chronic myeloid leukemia, is estimated to cost a patient $25,338 per year if the 400 mg tablets are taken by the patient once daily ($4,222.98 for a supply of 60 tablets [www.drugstore.com; accessed November 7, 2003]). A conservative estimate of the minimum cost of 6 months of maintenance treatment with epoetin alpha (Procrit), a biotech drug for the treatment of anemia in chemotherapy, is $1,867, assuming the patient uses the 2,000 units/ml vial 3 times per week, the lowest dosage available ($414.99 for a supply of 18 vials [www.drugstore.com; accessed November 7, 2003]).
As new targets emerge, and the rate of new drug introductions increases, institutions and governments will have to decide whether the benefits of these drugs are worth the higher costs. So incorporation of PE evaluations into formulary decision making for biotechnology drugs will definitely become critical. 8 For example, integration of monoclonal antibody (MoAb) products such as gemtuzumab (Mylotarg) or alemtuzumab (Campath) into the existing health care system is a challenge after being approved by the FDA in 2000 and 2001, respectively, because of their high acquisition costs: approximately $12,000 per course of therapy for gemtuzumab and $12,000 to $17,000 per month for alemtuzumab. MoAb products will need both proven clinical and economic profiles to support their place in the health care system. 19 Although hundreds of studies about CEA have been published, its present contribution in actual formulary decisions is still minor. Researchers have recently shown that CEA is rarely used to inform decisions about health services in the United States. 20 Although 72% of hospital pharmacy departments reported use of some CEA in formulary decision making, only 37% of them had the requisite CEA information available to them when considering a new drug for inclusion in a formulary because PE studies generally appeared in the postmarket phase when the new drug was already in the market. 15 Other surveys showed that, although CEA could have significant influence on formulary decisions made by pharmacy benefit managers (PBMs), health maintenance organizations (HMOs), and other health plans, some barriers impede the extensive application of CEA, such as inappropriate comparators, methodological issues in measuring costs and outcomes (e.g., what costs to be included, which effects to be measured, and how long to follow patients to determine these outcomes), lack of generalizability, concerns regarding study sponsorship, and lack of expertise for economic evaluation. 21,22 Although there is no exact number for what percentage of new drug evaluations for inclusion to formularies utilized CEA, it is obvious that the function of CEA was minor in pharmaceutical decision making in hospitals. 15 For HMOs and PBMs, market dynamics and the rise in drug expenditures demand greater efficiency and evaluation of medical care budgets that extend beyond the pharmacy budget only, increasing the importance and value of CEA for new, expensive drug therapies.

ss Limitations of Cost-Effectiveness Analysis
At present, there are significant limitations of CEA in formulary decision making. First, CEA is fundamentally limited to comparing a single outcome of a therapy or a single summary measure of related outcomes. Hence, the results hinge on the selection of the effect (outcome). Some diseases may have no distinct and unique measures that reflect the overall benefits/outcomes of drug therapies to serve as the indicator of outcomes. For example, reduction of gastroesophageal reflux disease symptoms may be used as the effect measure for a CEA, but this effect does not necessarily reflect how much patients really benefit. Another example is chemotherapy. Although chemotherapy may extend a patient' s life, its side effects may also severely impair the patient' s quality of life. Therefore, some PE studies tend to use more than 1 effect measure to capture treatment outcomes. For example, a study comparing the cost-effectiveness of antidepressants provided cost-effectiveness ratios for 2 outcome measures-symptom-free days and treatment success rates, which were defined as a more than 50% decline on scores of depression instruments such as Beck Depression Inventory and Hamilton Rating Scale of Depression, without relapse over a certain duration. 23 Similarly, a general or diseasespecific, health-related quality of life (HRQL) instrument score can be used in conjunction with clinical indicators to provide more information to the audiences.
CEA is simply a measure of production efficiency, not a measure of net gains or losses in welfare. 24 CEA can only identify the most efficient treatment, not whether the clinical outcomes gained are worth the cost of implementing the treatment. The most efficient treatment may still not be an acceptable use of resources, or treatments that look expensive in terms of the measured effect may produce unmeasured gains valued very highly by patients. Life-style drugs, such as sildenafil (Viagra) and minoxidil (Rogaine), are very expensive in terms of effect compared with life-saving drugs, but their benefits to the patients are highly valued and significantly impact reported quality of life.
A second limitation of CEA arises from inappropriate application of the tool. Some CEA studies compare only the new drug therapy with older therapies or even a placebo, 25 which can result in favorable incremental ratios simply by construction of the analysis. A close substitute in the same class or the most commonly used treatment would reflect the real value of the new drug and, therefore, be more informative to decision makers.
A third limitation is that timeliness in conducting PE analyses is often problematic in that a time lag for publication of studies makes them unavailable for formulary decisions when new drugs enter the market. Ideally, research should be completed before product launch and be available from the manufacturer if not yet published. The absence of reliable PE evaluations presents a major dilemma for P&T committees. Fortunately, many manufacturers are attempting to integrate PE research early in the drug development process. 26 Specifically, pharmaceutical companies can establish PE teams in Phase I clinical trials to gather relative economic information, conduct "cost-of-illness" studies, and formulate a PE model. In Phase II (small controlled clinical trials for safety and efficacy), preliminary PE research is then conducted, and in Phase III (large controlled or uncontrolled trials for efficacy), more extensive PE analyses are performed. Further, all the PE data gathered Cost-Effectiveness Analysis and the Formulary Decision-Making Process in Phases I, II, and III are analyzed and prepared for supporting the application for managed care organization (MCO) formulary listing. In this way, some valuable information can be gathered for a PE assessment by the P&T committee on a timely basis.
A fourth limitation is that drug company sponsors may make different modeling assumptions or give varying perspectives for the same set of drug therapies. The studies may have very different results and sometimes can be contradictory. Consequently, PBM and health plan decision makers must, at a minimum, perform their own independent assessment of the CEA model and reinterpret the findings. More commonly, the objectivity of manufacturer-sponsored CEA studies is called into question, which tends to result in dismissal of these studies in decision making. 25 Furthermore, it is sometimes recommended that CEA should take a social perspective rather than the view of segregated parties, such as patients, payers, or clinicians. 9 However, cost categories from a CEA study with a social perspective are broader than those from the perspective of a specific institution or health plan because, for example, providers may not consider patient costs or social costs. From a social perspective, costs would include time and costs that patients spend in the waiting room and transportation and even time and costs their families and friends spend in caring for the patient. However, these cost categories may not be as important from the payer' s perspective. For example, evaluation of the cost-effectiveness of peginterferon alpha-2b plus ribavirin versus interferon alpha-2b plus ribavirin for initial treatment of chronic hepatitis C from a social perspective showed that the ICER was €11,800 (euros) per QALY. 27 Decision makers from different interests, such as payers, may find information derived from a social perspective important but less so than the business interests associated with delivering the best possible care at an affordable price.
A fifth limitation is disagreement about what costs to include. Components generally included in CEA studies are direct costs (medical and nonmedical direct costs) and indirect costs reflecting productivity losses. However, health economists argue that, in some cases, productivity costs are implicitly included in the denominator of the cost-effectiveness ratio and therefore should not be added to the numerator. 28 29,30 The Human Capital method quantifies the total loss of productivity in terms of total forgone earnings, while the Friction Cost method measures the cost as lost earnings up to the time it takes for the worker to be replaced. Estimates using these 2 methods can be quite different, particularly for long illnesses and high-mortality illnesses. 29 One study found that the short-term indirect costs of back pain estimated by the Human Capital method were 3 times higher than the indirect costs estimated by the Friction Cost method. 30 Moreover, as discussed above, CEA typically leads to results presented in terms of ratios, i.e., average and incremental ratios. An inherent limitation of ratios is that they hide the underlying magnitude of the numerator and denominator. Hence, they do not inform decision makers regarding whether the treatment will have a significant overall impact on the budget.
Finally, PE models often rely on many assumptions, which may be incorrect or inappropriate. This has been one of the major concerns for decision makers and other readers. It is important to validate PE models whenever real-world data become available. Unfortunately, this has not been done frequently enough. A recent study reexamined 2 decision-analytic models for cost-effectiveness analysis comparing proton-pump inhibitor (PPI)-clarithromycin, or PPI-amoxicillin with bismuth-metronidazole-tetracycline (BMT) for Helicobacter pylori eradication in ulcer patients. 31 The original models heavily relied on assumptions and concluded that expensive PPI-based regimens were more cost effective than the relatively cheap BMT. In their study, Fairman and Motheral reassessed the analysis and found that a few key assumptions were inaccurate and biased the results, including discontinuation of antisecretory medication for all successfully treated patients, recurrent-associated utilization, inpatient care for all hospital care, and the degree of noncompliance. After adjusting the assumptions according to empirical data and expert opinion, the study found that the previous models overestimated the cost-effectiveness of PPI-based regimens and underestimated the cost-effectiveness of BMT. Therefore, reassessing the validity of PE models with empirical evidence is important for assuring the integrity and value of the PE information.
ss Developments Facilitating the Application of Cost-Effectiveness Analysis Some developments in various fields related to PEs have helped overcome some of the limitations of CEA and can work to make the method more applicable (Table 1).

Software
As mentioned above, formularies have been developed in many institutions, but local formulary decision makers may find it difficult to establish their own formularies using the published data because effectiveness and costs vary across different practice settings. In addition, some MCOs may find it difficult to utilize CEA studies with a social perspective.
An interactive computer program based on published CEA studies of hypercholesterolemia has been designed to enable users to adapt published data to their particular setting and perspective. 32 This software consists of 5 basic modules: a work sheet, data file, chart, macro program, and user interface. It allows the user to change all baseline data, such as efficacy data, drug costs, costs of side effects, physician and lab treatment protocol for each drug, and costs for physician and laboratory services. This model provides a generalized CEA framework for the specific health care institutions, such as HMOs and hospitals, and, therefore, is flexible and modifiable according to the particular end users and population. In addition, since the program does not require experience with computer-based tools, it is easier to use.
There are also other computer simulation models providing CEA on national or regional scales. [33][34][35] More such software modules will likely come onto the market in the future; there is surely a need for applicability to the unique settings and demographics of population groups. Modules that allow users to customize assumptions in a model to fit their setting, and thereby correct biases in costs and effects from "baseline" assumptions, are particularly needed.

Sensitivity Analysis
CEA involves many assumptions and variables with uncertain values, such as probabilities, life expectancies, discount rates, disease severity, target populations, etc. Sensitivity analysis can help in understanding the impact of uncertainty on the final results and ensures the validity of findings, such as how the change of one cost category affects the result. 12,36 However, conventional sensitivity analyses can sometimes lead to even greater confusion and misinterpretation. 37 For example, a cost-effectiveness study on rapid diagnostic testing followed by empiric antiviral therapy compared with no antiviral therapy for healthy adults with symptoms of influenza showed that the results were sensitive to influenza infection probability, proportion of type B influenza, the efficacy of antiviral drugs, and the value of a workday. 38 However, the sensitivity analysis was done by 1-way sensitivity analysis, i.e., varying the parameters one by one. The overall uncertainty was underestimated because the result depended on multiple parameters.
If more than 3 parameters, such as influenza infection probability, proportion of type B influenza, and the efficacy of antiviral drugs, are handled simultaneously in a multiple-way sensitivity analysis, the result would become very difficult or even impossible to interpret or illustrate with a graph. On the other hand, varying only a limited number of parameters at one time may not detect interactions between those parameters. Also, the selection of variables and alternative values for the variables to be included in a sensitivity analysis is subject to debate. Maximum and minimum values are commonly used to reflect the range of the analysis, but these values are very unlikely in real situations. Some explorations in statistics aimed to improve sensitivity analysis of CEA include probabilistic sensitivity analysis and rank-order stability analysis. 36,37

Probabilistic Sensitivity Analysis
Probabilistic sensitivity analysis is a method that considers uncertainties in all parameters simultaneously. 39 It assumes that each parameter has a range of possible values that follows a distribution function. The estimate of outcomes (costs or effects) is also a distribution function that depends on those of the individual parameters. The analysis is based on Monte Carlo simulation; it is used to compute point estimates and confidence intervals for the outcomes, such as mean costs, mean effects, and net health benefits. This approach has been discussed in detail by Shaw and Zachry. 37,39 Various software programs assist with conducting Monte Carlo simulation, such as TreeAge Software' s DATA 4.0 or DATA Pro. Though more comprehensive in its ability to predict the impact of uncertain parameters and to vary multiple parameters, its results still hinge on structural assumptions in the model. Furthermore, many of the inherent limitations of CEA, such as difficulty in capturing all outcomes and disagreement on cost categories to be included, are not resolvable through any form of sensitivity analysis.

Rank-Order Stability Analysis
Einarson et al. proposed an approach called the rank-order stability analysis (ROSA), which is a comprehensive and readily understandable method for validating results. 36 It is similar to a confidence-interval approach by providing ranges for all of the variables over which the results are valid. Compared with sensitivity analysis, ROSA is more comprehensive in that it provides intervals for all parameters under consideration and therefore improves the problem of incomplete PE analyses. The authors illustrated the steps of ROSA using an example of a PE analysis for drugs (drug A, B, and C) treating major depression and analyzed the impact of all factors to provide a comprehensive sensitivity analysis. ROSA provided upper and lower limits (where the rank order of ICER for the specified drug would change) for all parameters in the PE model, including duration of therapy, annual drug cost, annual medical care cost, and treatment success rate. Within the limits, the results were stable in the sense of what treatment should be chosen. Confidence intervals for parameters with available real-life data were calculated to further validate the robustness of the results. When the confidence intervals for the variables are located within the corresponding limits, the results were deemed stable.
Overall, ROSA reduces the potential for bias from incomplete conventional sensitivity analysis, which only selectively evaluates some, but not all, factors. However, further evaluation with this method may be necessary because this study only varied parameters one at a time while keeping other parameters constant. More meaningful information may be provided by manipulating multiple parameters simultaneously because parameters are often related to each other (i.e., in reality, 2 or more parameters may tend to move together).

Complementary Approaches
Two relatively new approaches-cost-consequence analysis (CCA) and budget impact analysis (BIA)-may help to reduce the barriers to the applicability of CEA in the formulary decision-making process by serving as complementary information to decision makers.

Cost-Consequence Analysis
Some researchers proposed this new approach to be used in the formulary decision-making process and believe it reflects the direction of future PEs. 40 Conceptually, CCA is a method in which costs and effects are calculated and listed as individual components but not aggregated into QALY or cost-effectiveness ratios. 41 Ideally, all relevant costs and health consequences, including direct costs, indirect costs, quality of life, QALYs, and clinical outcomes, are collected.
In addition, variation in the costs and effects across subpopulations are better presented, if available. For example, older patients may experience more side effects and less improvement than younger patients for a drug treatment. These differences in outcomes in age subpopulations should be listed. In effect, CCA includes inputs and outputs of one or more drug therapies as thoroughly as possible and then provides a cost-consequence tabulation created with the information collected for each therapy. These tabulations can be used to compare competing interventions. They can also be used as a basis for CEA. 41 An obvious advantage of CCA is that formulary decision makers can choose the resources, costs, and outcomes of interest to include in their economic evaluation. Since the cost-consequence list keeps all the information disaggregated, decision makers have to devise their own weight system to determine whether any additional health outcomes associated with the new drug deserves the extra cost incurred. For example, a CCA was developed by Paul et al. for the costs and consequences of an antiviral drug (acyclovir) compared with no treatment for varicella zosta virus infections, using data from a variety of sources. 42 The study provided a comprehensive range of costs and outcomes, including direct medical costs, productivity losses, and quality-of-life impact. The advantage of this analysis is that it allows readers to examine the data from different perspectives, such as from a managed care payer or from the societal standpoint. As an exercise, it would be worthwhile to involve P&T committee members in such greater PE sophistication.
Technically, for CEA to become more useful in formulary decisions, a combination of CEA and the usually disaggregated cost and outcome data found in CCA seems to be a promising approach. Pharmacoeconomic data submitted for formulary consideration should be presented in a cost-consequence list, which has all the cost categories and possible consequences listed separately. Clear and easy-to-understand summary results from CEA, including ICERs and sensitivity analyses, are provided accordingly. Such analyses may be better initiated by manufacturers alongside drug development to improve the reliability and timeliness of the analysis. It is also suggested that guidance from the FDA would be beneficial to reduce bias from the pharmaceutical teams that control the data; FDA guidance would provide consistency and oversight for providers to further trust the results.
Using this procedure, the limitations of CEA can be expected to be minimized. Decision makers who value CEA but would like more appropriate data may also find this approach useful because they can choose the information of most interest to perform their own CEA to closely reflect their concerns. 41 Nevertheless, such a procedure would still involve decisions being made based on comparing costs with specific measures of effect. Implicitly or explicitly, valuations will be put on the measured effects, and these should be scrutinized.
CCA has been used in various situations, including the evaluation of vaccines to provide estimates of the costs and benefits of different vaccine programs. For example, a cost-consequence analysis provided separate estimates of the direct medical care costs and productivity losses for a routine varicella vaccination program, which allows decision makers to assess their own budget impact and also the impact to society. 43 CCA was also used in the assessment of impacts for HIV infection treatment. A study evaluated the effect of adding lamivudine to treatment regimens containing zidovudine in patients with HIV infection and provided cost estimates for reduced HIV disease progression to AIDS or death, reduced number of hospital stays, unscheduled outpatient visits, and medications for HIV-related illness. 44 It showed that the lamivudine regimen had the potential to reduce the monthly costs associated with HIV-related illness and adverse events. Another study provided a list of costs and consequences associated with 2 drug treatments (zidovudine or zalcitabine) for patients with AIDS. 45 The data included health care utilization, functional status, quality of life, and work status. It showed that zidovudine had substantial advantages over zalcitabine in initial monotherapy of AIDS in terms of quality of life and resource utilization.

Budget Impact Analysis
Most CEA studies often fail to provide budget information with which the decision makers may be most concerned. However, manufacturer budget models could have built-in biases that should be checked. This could be done by examining whether and how the model includes appropriate cost categories and effects according to the model' s perspective. Health care decision makers usually have to consider the costs and benefits that fall within their own scope of responsibility, usually in terms of acquisition costs or budget "silos" to evaluate the impact of adding a new drug to the formulary. BIA, also called cost-impact analysis, provides an approach to evaluate the true financial impact of a new drug on the provider' s budget, an increasing concern with new drug introductions and rising overall drug costs. 46 BIA typically evaluates the total pharmacy costs incurred by adding 1 new drug into the formulary, from the purchaser' s perspective. For example, Meyer et al., using administrative claims data, estimated the incremental budget impact of a new interferon beta-1a product. 47 The per-member-per-month (PMPM) cost change for the addition of interferon beta-1a to a health plan with full injectable coverage and placement on tier 3 with prior authorization was $0.047. The authors concluded that such an incremental PMPM change would have minimal impact on the managed care pharmacy budget and, therefore, health care benefit managers would have flexibility in designing coverage for interferon beta-1a. However, since this study was conducted before the launch of the actual product, it has been criticized for not using complete information and making inappropriate assumptions related to relapse-free rate, costs of treating side effects, and incidence of neutralizing antibodies, etc. 48 As a result, it is necessary to update the BIA results using available empirical data to reflect the true impact of the drug.
CEA and BIA appear to complement each other to provide comprehensive cost and benefit information for formulary decision makers. For example, the decision maker can first rankorder drug alternatives according to the results from the CEA and then evaluate the budget impacts for treating different populations based on the budget constraints. It has been suggested by some national guidelines that BIA could be appropriate as a complementary approach for other PE analyses for the above consideration. [49][50][51][52] Evaluation of Pharmacoeconomic Literature Despite the improvement in the CEA methodologies, published studies are not equally valid and reliable. Thus, it is important to assess the quality of the PE literature to ensure that the P&T committee can at least differentiate papers with low and high quality and, accordingly, put a greater weight on better papers when making decisions. There are numerous guidelines, checklists, and criteria for the evaluation of PE literatures. Drummond' s framework for analysis is a popular checklist for qualitative assessment of literature. 53 54 The QHES includes 16 questions that are assigned weighted points according to their relative importance. PE analyses evaluated by the QHES will receive different scores according to their quality. This instrument gives decision makers an explicit and clear method to assign value to specific PE studies. The QHES has been shown to have good construct validity. 54 ss The Role of Guidelines for Cost-Effectiveness Analysis in Formulary Decision Making CEA guidelines are evidentiary standards for the provision of information to support clinical and economic evaluations of pharmaceuticals. Internationally, governments have driven the development of their own guidelines for PE evaluations, yet the FDA has confined itself to the marketing and promotional uses of such data. Creation of a Medicare drug benefit program could propel standardization of PE analyses and make these a requirement in formulary decision making. At the same time, private health systems in the United States are making a continuous effort to drive PE research based on the results of clinical trials toward naturalistic approaches, which are noncontrolled evaluation approaches that integrate drug assessments into routine or "natural" daily medical practice. 55 AMCP has been a leader in shaping standardization in formulary decision making.
Although CEA is being improved and complemented by various new technologies, the utilization of CEA cannot be guaranteed to be reliable without regulation. The advent of various guidelines provides standardized formats for PE evaluation, which may practically promote the application of CEA in formulary decision processes, as long as these guidelines are well structured and updated with the advance of science in PE, such as better ways of measuring costs, more flexible or accurate economic models, easy-to-use software, etc. Such guidelines, in their evolutionary development, must seek to transcend the interests of various parties and ultimately demonstrate that they are dedicated to public health improvements. For example, does the drug therapy improve outcomes in patients' quality of life, prevention of complications, and relief of the economic burden of disease on society?
Official guidelines have been adopted in countries other than the United States. Australia was the first country that required pharmaceutical companies seeking national formulary listing to provide a detailed economic analysis to support their case. 56 The Australian Guidelines for the Presentation of Submissions to the Pharmaceutical Benefits Advisory Committee came into effect on January 1993. Two other countries, Canada and New Zealand, have also made similar efforts. In Canada, the first edition of the Guidelines for Economic Evaluation of Pharmaceuticals: Canada was published in November 1994 and has been updated based on experiences and advances in the science. 57,58 The New Zealand guidelines came into effect on July 1, 1993. 54 Although guidelines raise the importance of economic evaluations for formulary approval, they need further refinement as PE improves as a science and as more experience is gained in its application. 56,57 It is important to recognize that advances in the field must be disseminated to the majority of managed care practitioners for proper implementation according to accepted standards in the field of PE outcomes research, such as criteria evolved and accepted in the International Society of Pharmacoeconomics and Outcomes Research.
In the United States, although the FDA has not required PE data as part of new drug applications, results of PE studies have become more likely to be required by P&T committees of provider institutions and MCOs in drug formulary decisions. 59 In 2001, AMCP published its Format for Formulary Submissions. 60 Since that time, it has become common practice for pharmaceutical manufacturers to submit clinical and economic dossiers for formulary approval. 10 The AMCP Format requires data-driven economic evaluations, including prospective cost efficacy, or cost-effectiveness studies, as well as retrospective economic evaluations, along with literature reviews. PE models are used to examine the impact of uncertainty in the estimates of treatment effectiveness and resources consumed by each treatment process. These models can support the formulary decision by providing information, such as total costs, total effectiveness, and incremental cost-effectiveness ratios of the drug and its appropriate comparator products. A number of drug manufacturers have endorsed this step and are working with MCOs in its speedy implementation; they see the guidelines fostering communication between plans and manufacturers around evidence-based materials. 61 The AMCP Format requests that the manufacturer identify all relevant PE studies for the product and provide a justification of studies for the population being served. Electronic copies of spreadsheets and models are to be submitted so that the P&T committee can rework the data, which is common to determine what areas of uncertainty have major impacts on projected costs and outcomes. Comparisons of studies should reveal limitations and help identify model deficiencies or poor applicability. Such a review will improve competency in PE among firms that have not fully developed their capability. Disease-management strategies that are recommended by the manufacturer are also requested, which gives further opportunities for the MCO to evaluate the degree of integration of clinical and economic factors for the use of the product.
By establishing standardized PE evaluation methods, the AMCP guidelines will spur such PE data for MCOs and may likely spill over for other health care providers, such as hospitals and nursing homes. Cost-Effectiveness Analysis and the Formulary Decision-Making Process choices for drug formulary lists. Particularly in consideration of worldwide concern for dramatically rising drug expenditures, it is clearly a very worthwhile advance to have a more firm scientific footing for preferred pharmacotherapeutic choices by providers and governments. Yet, the development and application of PE is still in a nascent phase, which will necessitate continual improvements in designs, methodologies, and applications. As PE matures, a primary concern should be the usefulness of the information that is reported to actual decision makers.
In the short term, when doing a CEA, a BIA can be done from the purchaser' s perspective to provide complementary information to decision makers. Including a BIA with CEA will help to increase the applicability and practical usefulness of CEA, thus facilitating the formulary decision-making process. Standard guidelines regarding PE analysis must advance to improve CEA' s relevance and reliability and thus provide a more scientific basis for formulary decision processes.
Strengthening PE requirements may exert an additional cost to sponsors and researchers. However, by imposing a standardized approach of reporting cost-effectiveness and CCA, such studies would offer more complete, accurate, reliable, and useful information. In this regard, the AMCP guidelines represent a robust effort to spur the applicability and practicality of CEA. Already, the drug dossiers requested from the manufacturers ask for baseline estimates of resource utilization and costs imposed by new drug therapies, and most U.S. drug manufacturers are complying appropriately to obtain and deliver these data. Scenarios and assumptions specific to a health plan' s patient population (e.g., projected for 3 years) allow for a realistic assessment of the aggregate cost impact associated with new treatments. The pharmaceutical industry is increasingly embracing economic modeling techniques, utilizing explicit and recognizable mathematical bases, to incorporate the best available evidence. Standardization demanded by the AMCP Format will result in greater trust and respect for the PE analyses presented by pharmaceutical manufacturers in drug product dossiers, increasing the value and use by MCOs in formulary decision making.